Skip to content

feat: support model fallback endpoints - #1083

Merged
yaozheng-fang merged 1 commit into
volcengine:mainfrom
xgtcode:feat/model-fallback-endpoint
Sep 9, 2026
Merged

feat: support model fallback endpoints#1083
yaozheng-fang merged 1 commit into
volcengine:mainfrom
xgtcode:feat/model-fallback-endpoint

Conversation

@xgtcode

@xgtcode xgtcode commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds framework-level support for model fallback endpoint configuration.

It introduces ModelFallbackEndpoint so Agent(model_fallbacks=...) can accept fallback models with independent provider, API base, API key, and LiteLLM extra config. It also exports the new type from veadk, wires fallback configs into RetryingLiteLlm, and adds runtime compatibility warnings for runtimes that do not support explicit model fallbacks.

Changes

  • Add ModelFallbackEndpoint and ModelFallbackConfig.
  • Support same-provider string fallbacks and cross-provider endpoint fallbacks in Agent.
  • Export ModelFallbackEndpoint from veadk.__init__.
  • Pass structured fallback configs through RetryingLiteLlm.
  • Warn when external runtimes drop Agent(model_fallbacks=...).
  • Add unit coverage for fallback model behavior and runtime compatibility messaging.

Testing

  • uv run pre-commit run --files veadk/__init__.py veadk/agent.py veadk/models/retrying_lite_llm.py veadk/runtime/compat.py tests/test_agent.py tests/runtime/differential/test_runtime_parity.py
  • uv run pytest tests/test_agent.py tests/runtime/differential/test_runtime_parity.py -q

Result: 44 passed, 2 xfailed

@yaozheng-fang
yaozheng-fang merged commit 25a8ea6 into volcengine:main Sep 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants